{
  "cells": [
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "txu0M8tPKoLh"
      },
      "source": [
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pinecone-io/examples/blob/master/learn/analytics-and-ml/extreme-classification/extreme-classification.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/pinecone-io/examples/blob/master/learn/analytics-and-ml/extreme-classification/extreme-classification.ipynb)\n",
        "\n",
        "# Extreme Classification\n",
        "\n",
        "This demo aims to label new texts automatically when the number of possible labels is enormous. This scenario is known as extreme classification, a supervised learning variant that deals with multi-class and multi-label problems involving many choices. \n",
        "\n",
        "Examples for applying extreme classification are labeling a new article with Wikipedia's topical labels, matching web content with a set of relevant advertisements, classifying product descriptions with catalog labels, and classifying a resume into a collection of pertinent job titles. \n",
        "\n",
        "Here's how we'll perform extreme classification:\n",
        "\n",
        "1. We'll transform 250,000 labels into vector embeddings using a publicly available embedding model and upload them into a [managed vector index](https://www.pinecone.io/). \n",
        "2. Then we'll take an article that requires labeling and transform it into a [vector embedding](https://www.pinecone.io/learn/vector-embeddings/) using the same model.\n",
        "3. We'll use that article's vector embedding as the query to search the vector index. In effect, this will retrieve the most similar labels to the article's semantic content.\n",
        "4. With the most relevant labels retrieved, we can automatically apply them to the article.\n",
        "\n",
        "Let's get started!"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "XqdwjnhLKoLi"
      },
      "source": [
        "# Install Dependencies"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "J7vHPDeAw4yv"
      },
      "outputs": [],
      "source": [
        "!pip install -qU pinecone-client==3.1.0 sentence-transformers datasets"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "Rehgtr0exfNC"
      },
      "source": [
        "# Setting up Pinecone's Similarity Search Service\n",
        "Here we set up our similarity search service. We assume you are familiar with Pinecone's [quick start tutorial](https://www.pinecone.io/docs/quickstart-python/). To create our vector index, we first need to initialize our connection to Pinecone. For this we need a [free API key](https://app.pinecone.io). Once we have that, we initialize the connection like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import os\n",
        "from pinecone import Pinecone\n",
        "\n",
        "# initialize connection to pinecone (get API key at app.pinecone.io)\n",
        "api_key = os.environ.get('PINECONE_API_KEY') or 'PINECONE_API_KEY'\n",
        "\n",
        "# configure client\n",
        "pc = Pinecone(api_key=api_key)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now we setup our index specification, this allows us to define the cloud provider and region where we want to deploy our index. You can find a list of all [available providers and regions here](https://docs.pinecone.io/docs/projects)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from pinecone import ServerlessSpec\n",
        "\n",
        "cloud = os.environ.get('PINECONE_CLOUD') or 'aws'\n",
        "region = os.environ.get('PINECONE_REGION') or 'us-east-1'\n",
        "\n",
        "spec = ServerlessSpec(cloud=cloud, region=region)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "SSwTmV2VikI2"
      },
      "source": [
        "Now we create a new index called \"extreme-ml\". What we name this isn't important."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "index_name = 'extreme-ml'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import time\n",
        "\n",
        "# check if index already exists (it shouldn't if this is first time)\n",
        "if index_name not in pc.list_indexes().names():\n",
        "    # if does not exist, create index\n",
        "    pc.create_index(\n",
        "        index_name,\n",
        "        dimension=384,\n",
        "        metric='cosine',\n",
        "        spec=spec\n",
        "    )\n",
        "    # wait for index to be initialized\n",
        "    while not pc.describe_index(index_name).status['ready']:\n",
        "        time.sleep(1)\n",
        "\n",
        "# connect to index\n",
        "index = pc.Index(index_name)\n",
        "# view index stats\n",
        "index.describe_index_stats()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "4vrtoBXs1Yer"
      },
      "source": [
        "# Data Preparation \n",
        "In this demo, we classify Wikipedia articles using a standard dataset from an extreme classification benchmarking [resource](http://manikvarma.org/downloads/XC/XMLRepository.html). The data used in this example is [Wikipedia-500k](https://drive.google.com/drive/folders/12HiiGWmbLfTEEObs2Y2jiTETZfXDowrn) which contains around 500,000 labels. We will load a subset of this dataset from huggingface which contains 200,000 articles with around 250,000 different labels, already prepared for this classification task.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 293,
          "referenced_widgets": [
            "5ec13551e13d43c38cce4fb3d3edbe87",
            "c808ed22f673424f8e0674398af4a5f3",
            "7a5c95f6c8e0496eb794cdd41861deb3",
            "f5b580095caf4cd1afea29cf9f5eb701",
            "6d465e6260cb47af97a261154e565392",
            "0bd943a43b9f468aadd1b3f922b3b26e",
            "82f52ab4f7094e5bba31fc8e49edc322",
            "46919d2bad1d45c9b314a9269063906f",
            "64d43e26c9e94e3aab0a98d188ad3643",
            "d8c31a57a0474fe8aee03b67076dcff5",
            "be3fe277d856487a93303ca1cd1a233a"
          ]
        },
        "id": "mNoq5sQ-KoLp",
        "outputId": "c5049002-0d58-4481-ba74-74ffd54dc971"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "WARNING:datasets.builder:Using custom data configuration ashraq--WikiTitles-200K-b2afe09cf8d0e2ee\n",
            "WARNING:datasets.builder:Reusing dataset parquet (/root/.cache/huggingface/datasets/ashraq___parquet/ashraq--WikiTitles-200K-b2afe09cf8d0e2ee/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec)\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "5ec13551e13d43c38cce4fb3d3edbe87",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/2 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-1e4fed8a-9e4b-45c8-893f-7a628a2575a5\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>title</th>\n",
              "      <th>content</th>\n",
              "      <th>target</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Anarchism</td>\n",
              "      <td>anarchism is a political philosophy that a...</td>\n",
              "      <td>[Anarchism, Anti-capitalism, Anti-fascism, Far...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Academy_Awards</td>\n",
              "      <td>the academy awards or the oscars (the offi...</td>\n",
              "      <td>[1929_establishments_in_the_United_States, Aca...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Anthropology</td>\n",
              "      <td>anthropology  is the scientific study of hu...</td>\n",
              "      <td>[Anthropology, Social_sciences]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>American_Football_Conference</td>\n",
              "      <td>the american football conference (afc) is o...</td>\n",
              "      <td>[American_Football_League, National_Football_L...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Analysis_of_variance</td>\n",
              "      <td>analysis of variance (anova) is a collection ...</td>\n",
              "      <td>[Analysis_of_variance, Design_of_experiments, ...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-1e4fed8a-9e4b-45c8-893f-7a628a2575a5')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-1e4fed8a-9e4b-45c8-893f-7a628a2575a5 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-1e4fed8a-9e4b-45c8-893f-7a628a2575a5');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "                          title  \\\n",
              "0                     Anarchism   \n",
              "1                Academy_Awards   \n",
              "2                  Anthropology   \n",
              "3  American_Football_Conference   \n",
              "4          Analysis_of_variance   \n",
              "\n",
              "                                             content  \\\n",
              "0      anarchism is a political philosophy that a...   \n",
              "1      the academy awards or the oscars (the offi...   \n",
              "2     anthropology  is the scientific study of hu...   \n",
              "3     the american football conference (afc) is o...   \n",
              "4   analysis of variance (anova) is a collection ...   \n",
              "\n",
              "                                              target  \n",
              "0  [Anarchism, Anti-capitalism, Anti-fascism, Far...  \n",
              "1  [1929_establishments_in_the_United_States, Aca...  \n",
              "2                    [Anthropology, Social_sciences]  \n",
              "3  [American_Football_League, National_Football_L...  \n",
              "4  [Analysis_of_variance, Design_of_experiments, ...  "
            ]
          },
          "execution_count": 22,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from datasets import load_dataset\n",
        "\n",
        "# load the dataset from huggingface\n",
        "data = load_dataset(\"ashraq/WikiTitles-200K\")\n",
        "# load the train split into a pandas dataframe\n",
        "df = data[\"train\"].to_pandas()\n",
        "df.head()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "bO0Jk_5YYhEX"
      },
      "source": [
        "# Create Vector Embeddings"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "-YZh1YM2KoLw"
      },
      "source": [
        "Recall, we want to index and search all possible (250,000) *labels*. We do that by averaging, for each label, the corresponding article vector embeddings that contain that label. \n",
        "\n",
        "Let's first create the article vector embeddings. Here we use the SentenceTransformer model [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) to generate article vector embeddings."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Kn6xK4c0E5DQ",
        "outputId": "4e55fde1-07a2-4e04-87ca-dfdd9603683e"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "SentenceTransformer(\n",
              "  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel \n",
              "  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})\n",
              "  (2): Normalize()\n",
              ")"
            ]
          },
          "execution_count": 5,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from sentence_transformers import SentenceTransformer\n",
        "import torch\n",
        "\n",
        "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
        "\n",
        "# load the model from huggingface\n",
        "model = SentenceTransformer(\n",
        "    'sentence-transformers/all-MiniLM-L6-v2',\n",
        "    device=device\n",
        ")\n",
        "model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "d3dd4f08680043ccac481d8c2b14b146",
            "53bc873723064c908519d9bd06c6a7b6",
            "d17279116b8543b8ba730efbcc9c8726",
            "356b7c213dd5471ea128b88664e2a5c8",
            "3540368938024ed4b9d5f9b737ac7991",
            "7e29a99abd6545dd84d7881be9f3127c",
            "ca9c561bc7084d4486c4a47e69d06aeb",
            "8c99a7c4d3804ea1889279f2399dc57b",
            "a5fb49b58ae6474a894f994426cbc119",
            "2c00cdae533d4ce891a5e3bbae019850",
            "506554b524654f65b65ac6a6994ba763"
          ]
        },
        "id": "dHSYVldeFN0i",
        "outputId": "5539dda7-1f3b-48ee-fb7d-6f810e0a1ddc"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "d3dd4f08680043ccac481d8c2b14b146",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Batches:   0%|          | 0/6250 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import pandas as pd\n",
        "\n",
        "# Create embeddings\n",
        "encoded_articles = model.encode(df['content'].tolist(), show_progress_bar=True)\n",
        "# add the embeddings to our dataframe\n",
        "df['content_vector'] = pd.Series(encoded_articles.tolist())"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "e3GNhDdaKoLx"
      },
      "source": [
        "It appears that using the article embeddings per se doesn't provide good enough accuracies. Therefore, we chose to index and search the labels directly. The label embedding is simply the average of all its corresponding article embeddings.\n",
        "\n",
        "Let's create the label embeddings."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 394
        },
        "id": "WTMqqIJROS2S",
        "outputId": "1820c2cf-5bcf-4b0f-f42d-f22dcabc9566"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-ff91400d-6a95-4b89-80aa-5abe24b63896\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>content_vector</th>\n",
              "      <th>label</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>target</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>Chilean_people_of_Spanish_descent</th>\n",
              "      <td>[-0.01580731357846941, -0.0031899410699095044,...</td>\n",
              "      <td>Chilean_people_of_Spanish_descent</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Danish_scientists</th>\n",
              "      <td>[-0.0458383746445179, 0.06631474941968918, -0....</td>\n",
              "      <td>Danish_scientists</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Jewish_delicatessens</th>\n",
              "      <td>[0.004267878830432892, 0.019741827622056007, -...</td>\n",
              "      <td>Jewish_delicatessens</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Companies_based_in_Victoria,_British_Columbia</th>\n",
              "      <td>[-0.009467452298849821, -0.014646705240011215,...</td>\n",
              "      <td>Companies_based_in_Victoria,_British_Columbia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cities_in_Lewis_County,_Kentucky</th>\n",
              "      <td>[0.04348186030983925, -0.012287872843444347, 0...</td>\n",
              "      <td>Cities_in_Lewis_County,_Kentucky</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Airlines_of_Costa_Rica</th>\n",
              "      <td>[0.07663661427795887, 0.02771153673529625, -0....</td>\n",
              "      <td>Airlines_of_Costa_Rica</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>School_buildings_completed_in_1891</th>\n",
              "      <td>[0.07744941860437393, -0.017717836424708366, -...</td>\n",
              "      <td>School_buildings_completed_in_1891</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Political_parties_established_in_1971</th>\n",
              "      <td>[-0.02586129680275917, -0.039618962444365025, ...</td>\n",
              "      <td>Political_parties_established_in_1971</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1946_in_Japan</th>\n",
              "      <td>[-0.026875406503677368, 0.03590376675128937, 0...</td>\n",
              "      <td>1946_in_Japan</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19th_century_in_Rome</th>\n",
              "      <td>[-0.09453806281089783, -0.02094138227403164, 0...</td>\n",
              "      <td>19th_century_in_Rome</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ff91400d-6a95-4b89-80aa-5abe24b63896')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-ff91400d-6a95-4b89-80aa-5abe24b63896 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-ff91400d-6a95-4b89-80aa-5abe24b63896');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "                                                                                  content_vector  \\\n",
              "target                                                                                             \n",
              "Chilean_people_of_Spanish_descent              [-0.01580731357846941, -0.0031899410699095044,...   \n",
              "Danish_scientists                              [-0.0458383746445179, 0.06631474941968918, -0....   \n",
              "Jewish_delicatessens                           [0.004267878830432892, 0.019741827622056007, -...   \n",
              "Companies_based_in_Victoria,_British_Columbia  [-0.009467452298849821, -0.014646705240011215,...   \n",
              "Cities_in_Lewis_County,_Kentucky               [0.04348186030983925, -0.012287872843444347, 0...   \n",
              "Airlines_of_Costa_Rica                         [0.07663661427795887, 0.02771153673529625, -0....   \n",
              "School_buildings_completed_in_1891             [0.07744941860437393, -0.017717836424708366, -...   \n",
              "Political_parties_established_in_1971          [-0.02586129680275917, -0.039618962444365025, ...   \n",
              "1946_in_Japan                                  [-0.026875406503677368, 0.03590376675128937, 0...   \n",
              "19th_century_in_Rome                           [-0.09453806281089783, -0.02094138227403164, 0...   \n",
              "\n",
              "                                                                                       label  \n",
              "target                                                                                        \n",
              "Chilean_people_of_Spanish_descent                          Chilean_people_of_Spanish_descent  \n",
              "Danish_scientists                                                          Danish_scientists  \n",
              "Jewish_delicatessens                                                    Jewish_delicatessens  \n",
              "Companies_based_in_Victoria,_British_Columbia  Companies_based_in_Victoria,_British_Columbia  \n",
              "Cities_in_Lewis_County,_Kentucky                            Cities_in_Lewis_County,_Kentucky  \n",
              "Airlines_of_Costa_Rica                                                Airlines_of_Costa_Rica  \n",
              "School_buildings_completed_in_1891                        School_buildings_completed_in_1891  \n",
              "Political_parties_established_in_1971                  Political_parties_established_in_1971  \n",
              "1946_in_Japan                                                                  1946_in_Japan  \n",
              "19th_century_in_Rome                                                    19th_century_in_Rome  "
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import numpy as np\n",
        "\n",
        "# Explode the target indicator column\n",
        "df_explode = df.explode('target')\n",
        "\n",
        "# Group by label and define a unique vector for each label\n",
        "label_vectors = df_explode.groupby('target').agg(mean=('content_vector', lambda x: np.vstack(x).mean(axis=0).tolist()))\n",
        "label_vectors['target'] = label_vectors.index\n",
        "label_vectors.columns = ['content_vector', 'label']\n",
        "\n",
        "label_vectors.sample(10)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "-uVBrkZ95Yhq"
      },
      "source": [
        "# Upsert Embeddings to Pinecone Index"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "79a17d61089740258ee52d59c3db9600",
            "2b277671c7ea43d6a68b66b0bd3fc487",
            "d7e69bd705104472ac10d4c24b4fb874",
            "362cc689a3034fb29bb9a3e64651117d",
            "8edf72ce4a9b4fb697c205dfe9210ef5",
            "63bd9373e73d4b11bf8c182ae34035c7",
            "326632625859494683473020a7cfa2e1",
            "cdc5dd43518f4cba8647343fbf59053a",
            "b8f1e5f97f364aeab22bf6f991290f78",
            "eba140aca7da4c9ab9e3f2734693dd44",
            "4a904bf964b6407d83443dbeacd7aebd"
          ]
        },
        "id": "YDhOKf__61Oh",
        "outputId": "10b976f2-8c3a-45a4-8a24-ecb1be37cb77"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "79a17d61089740258ee52d59c3db9600",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/1004 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "from tqdm.auto import tqdm\n",
        "\n",
        "# we will use batches of 256\n",
        "batch_size = 256\n",
        "\n",
        "for i in tqdm(range(0, len(label_vectors), batch_size)):\n",
        "    # find end of batch\n",
        "    i_end = min(i+batch_size, len(label_vectors))\n",
        "    # extract batch\n",
        "    batch = label_vectors.iloc[i:i_end]\n",
        "    # select embeddings for batch\n",
        "    emb = batch[\"content_vector\"].tolist()\n",
        "    # get metadata\n",
        "    meta = [{\"label\": l} for l in batch[\"label\"]]\n",
        "    # create unique IDs\n",
        "    ids = [f\"{idx}\" for idx in range(i, i_end)]\n",
        "    # add all to upsert list\n",
        "    to_upsert = list(zip(ids, emb, meta))\n",
        "    # upsert/insert these records to pinecone\n",
        "    _ = index.upsert(vectors=to_upsert)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "-GVhg9LvKoLy"
      },
      "source": [
        "Let's validate the number of indexed labels."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "83fitZsgJ0p8",
        "outputId": "e68f2df1-9234-44eb-d0c0-4ec98a06b78a"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'dimension': 384,\n",
              " 'index_fullness': 0.2,\n",
              " 'namespaces': {'': {'vector_count': 256899}},\n",
              " 'total_vector_count': 256899}"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# check that we have all vectors in index\n",
        "index.describe_index_stats()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "XfXXbYZ8xsZe"
      },
      "source": [
        "# Query "
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "13buVJS4KoLz"
      },
      "source": [
        "Now, let's test the vector index and examine the classifier results. We use new articles which were not used when we generated the label embeddings. For that, we load the test split of the dataset as follows:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "_Zhu8jxA0Obn",
        "outputId": "432ee174-c90c-4798-d09c-4fec7215d1ea"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-b527d4d7-4d87-42ec-bd37-bf50c09939d4\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>title</th>\n",
              "      <th>content</th>\n",
              "      <th>target</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Autism</td>\n",
              "      <td>&lt;!-- notes: 1) please follow the wikipedia...</td>\n",
              "      <td>[Autism, Communication_disorders, Mental_and_b...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Altruism</td>\n",
              "      <td>altruism or selflessness is the principle o...</td>\n",
              "      <td>[Altruism, Auguste_Comte, Defence_mechanisms, ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Alchemy</td>\n",
              "      <td>alchemy is an influential tradition whose p...</td>\n",
              "      <td>[Alchemy, Esotericism, Hermeticism]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Andorra</td>\n",
              "      <td>{{infobox country | conventional_long_name ...</td>\n",
              "      <td>[Andorra, Constitutional_monarchies, Countries...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Alkali_metal</td>\n",
              "      <td>the alkali metals are a group (column) in th...</td>\n",
              "      <td>[Alkali_metals, Groups_in_the_periodic_table, ...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b527d4d7-4d87-42ec-bd37-bf50c09939d4')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-b527d4d7-4d87-42ec-bd37-bf50c09939d4 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-b527d4d7-4d87-42ec-bd37-bf50c09939d4');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "          title                                            content  \\\n",
              "0        Autism      <!-- notes: 1) please follow the wikipedia...   \n",
              "1      Altruism     altruism or selflessness is the principle o...   \n",
              "2       Alchemy     alchemy is an influential tradition whose p...   \n",
              "3       Andorra     {{infobox country | conventional_long_name ...   \n",
              "4  Alkali_metal    the alkali metals are a group (column) in th...   \n",
              "\n",
              "                                              target  \n",
              "0  [Autism, Communication_disorders, Mental_and_b...  \n",
              "1  [Altruism, Auguste_Comte, Defence_mechanisms, ...  \n",
              "2                [Alchemy, Esotericism, Hermeticism]  \n",
              "3  [Andorra, Constitutional_monarchies, Countries...  \n",
              "4  [Alkali_metals, Groups_in_the_periodic_table, ...  "
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# load the test split into a pandas dataframe\n",
        "df_test = data[\"test\"].to_pandas()\n",
        "df_test.head()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "voSjI7hWneiy"
      },
      "source": [
        "First, we write some helper functions to select test articles and to query pinecone index for labels."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "z8n0OgLmIesV"
      },
      "outputs": [],
      "source": [
        "from pprint import pprint"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "99ujDnLMmvLc"
      },
      "outputs": [],
      "source": [
        "def select_test_article(index):\n",
        "    print(\"Query Article:\")\n",
        "    # select the article associated with index from test split\n",
        "    article = df_test.iloc[index]\n",
        "    # print test article data\n",
        "    data = {\"Title\": article.title, \"Content\": article.content[:1000],  \"Original Labels\": list(article.target)}\n",
        "    pprint(data)\n",
        "    return article"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "LNzPqjGKfB8Z"
      },
      "outputs": [],
      "source": [
        "def query_pinecone(article, top_k=3):\n",
        "    # Create embeddings for test articles\n",
        "    xq = model.encode(article.content).tolist()\n",
        "    # query pinecone for labels\n",
        "    results = index.query(vector=xq, top_k=top_k, include_metadata=True)\n",
        "    # select only the labels from result and print\n",
        "    labels = [res[\"metadata\"][\"label\"] for res in results.matches]\n",
        "    pprint({\"Predicted Labels\": labels})"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "6lEtWHZB0_ik"
      },
      "source": [
        "Now let's run some queries. We select articles from the test split and use its content embeddings to query the Pinecone index and retrieve the most similar labels to the article's semantic content."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WQjq29fvAsL8",
        "outputId": "33b16a40-58bb-4aab-ad56-cdac0b252365"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Query Article:\n",
            "{'Content': 'mpeg-4 is a method of defining compression of audio and visual '\n",
            "            '(av) digital data. it was introduced in late 1998 and designated '\n",
            "            'a standard for a group of audio and video coding formats and '\n",
            "            'related technology agreed upon by the iso/iec moving picture '\n",
            "            'experts group (mpeg) (iso/iec jtc1/sc29/wg11) under the formal '\n",
            "            'standard iso/iec 14496&nbsp;\u00e2\\x80\\x93 coding of audio-visual '\n",
            "            'objects. uses of mpeg-4 include compression of av data for web '\n",
            "            '(streaming media) and cd distribution, voice (telephone, '\n",
            "            'videophone) and broadcast television applications. ==background== '\n",
            "            'mpeg-4 absorbs many of the features of mpeg-1 and mpeg-2 and '\n",
            "            'other related standards, adding new features such as (extended) '\n",
            "            'vrml support for 3d rendering, object-oriented composite files '\n",
            "            '(including audio, video and vrml objects), support for externally '\n",
            "            'specified digital rights management and various types of '\n",
            "            'interactivity. aac (advanced audio coding) was standardized as an '\n",
            "            'adjunct to mpeg-2 (as part 7) before mpeg-4 was issued. mpeg-4 is '\n",
            "            'sti',\n",
            " 'Original Labels': ['Audio_codecs',\n",
            "                     'MPEG-4',\n",
            "                     'Video_codecs',\n",
            "                     'Videotelephony'],\n",
            " 'Title': 'MPEG-4'}\n"
          ]
        }
      ],
      "source": [
        "article = select_test_article(524)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6YvyFKw2pDpW",
        "outputId": "d1fa2aca-2190-4654-e8cf-ae435bef9349"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{'Predicted Labels': ['Container_formats',\n",
            "                      'MPEG',\n",
            "                      'Open_standards_covered_by_patents',\n",
            "                      'Video_codecs',\n",
            "                      'Microsoft_Windows_multimedia_technology',\n",
            "                      'Free_video_software',\n",
            "                      'Audio_format_converters',\n",
            "                      'Video_conversion_software',\n",
            "                      'Video_editing_software',\n",
            "                      'Digital_rights_management_standards']}\n"
          ]
        }
      ],
      "source": [
        "query_pinecone(article, top_k=10)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "2eWFXptk55_U"
      },
      "source": [
        "Let's run some more queries."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "J7t0REo3xbWL",
        "outputId": "9cd39fbc-7114-4d63-a46f-2f45a722b0df"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Query Article:\n",
            "{'Content': '  the houston astros are a professional baseball team located in '\n",
            "            'houston, texas. the team is a member of the western division of '\n",
            "            \"major league baseball's american league, having moved in 2013 \"\n",
            "            'after spending their first 51 seasons in the national league. the '\n",
            "            'astros have played their home games at minute maid park since '\n",
            "            '2000. the astros were established as the houston colt .45s in . '\n",
            "            'th',\n",
            " 'Original Labels': ['Houston_Astros',\n",
            "                     'Major_League_Baseball_teams',\n",
            "                     'Professional_baseball_teams_in_Texas',\n",
            "                     'Sports_clubs_established_in_1962'],\n",
            " 'Title': 'Houston_Astros'}\n"
          ]
        }
      ],
      "source": [
        "article = select_test_article(354)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FEiGpiW2DlDI",
        "outputId": "ee006948-227a-4e96-fd62-332c29998c50"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{'Predicted Labels': ['1965_Major_League_Baseball_season',\n",
            "                      'Houston_Astros_seasons',\n",
            "                      'Cities_in_Texas_County,_Missouri',\n",
            "                      'San_Diego_Padres_minor_league_affiliates',\n",
            "                      'Professional_baseball_teams_in_Texas',\n",
            "                      'Sports_in_San_Antonio,_Texas',\n",
            "                      'Houston_Astros_broadcasters',\n",
            "                      'Texas_soccer_clubs',\n",
            "                      'Sports_venues_in_Greater_Orlando',\n",
            "                      'Buildings_and_structures_in_Kissimmee,_Florida',\n",
            "                      'Martinsville_Astros_players',\n",
            "                      'Culture_of_Houston,_Texas',\n",
            "                      'American_soccer_clubs_2008_season',\n",
            "                      '2008_Major_League_Soccer_season',\n",
            "                      'Buildings_and_structures_in_Houston,_Texas']}\n"
          ]
        }
      ],
      "source": [
        "query_pinecone(article, top_k=15)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8uuofl0WFPfR",
        "outputId": "bddd79dc-fb5c-4a19-fb59-cadcf27ed6c8"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Query Article:\n",
            "{'Content': ' this article details the variety of means of transport in '\n",
            "            'jersey, channel islands. thumb|cycle lane in st helier ==air '\n",
            "            'transport== airports: *jersey airport ==rail transport== '\n",
            "            'historically there were public railway services in the island, '\n",
            "            'provided by two railway companies: *the jersey railway closed in '\n",
            "            '1936. *the jersey eastern railway closed in 1929. during the '\n",
            "            'german military occupation 1940\u00e2\\x80\\x931945, light railways were '\n",
            "            're-established by the germans for the purpose of supplying '\n",
            "            'coastal fortifications. a one-metre gauge line was laid down '\n",
            "            'following the route of the former jersey railway from saint '\n",
            "            'helier to la corbi\u00c3\u00a8re, with a branch line connecting the stone '\n",
            "            'quarry at ronez in saint john. a 60cm line ran along the west '\n",
            "            'coast, and another was laid out heading east from saint helier to '\n",
            "            'gorey. the first line was opened in july 1942, the ceremony being '\n",
            "            'disrupted by passively resisting jersey spectators.cruickshank, '\n",
            "            'charles g. (1975) the german occupation of the channel islands, '\n",
            "            'the guer',\n",
            " 'Original Labels': ['Ports_and_harbours_of_Jersey',\n",
            "                     'Transport_in_British_Overseas_Territories_and_Crown_Dependencies',\n",
            "                     'Transport_in_Jersey'],\n",
            " 'Title': 'Transport_in_Jersey'}\n"
          ]
        }
      ],
      "source": [
        "article = select_test_article(400)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2vXtn1UnGAZN",
        "outputId": "18d97f47-5235-49f9-a1f9-ec78a67d4de3"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{'Predicted Labels': ['PATH_stations_in_New_Jersey',\n",
            "                      'New_Jersey_Transit_Rail_Operations',\n",
            "                      'New_Jersey_streetcar_lines',\n",
            "                      'Hudson-Bergen_Light_Rail_stations',\n",
            "                      'Beeching_closures_in_Wales',\n",
            "                      'Railway_stations_opened_in_1898',\n",
            "                      'Rail_infrastructure_in_New_Jersey',\n",
            "                      'Transportation_in_Rockland_County,_New_York',\n",
            "                      'Closed_railway_lines_in_Wales',\n",
            "                      'Disused_railway_stations_in_Kent']}\n"
          ]
        }
      ],
      "source": [
        "query_pinecone(article, top_k=10)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "uCnTyEMPSHPj"
      },
      "source": [
        "We can see that the predicted labels are in fact related to the articles."
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Summary"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "qI68OoTiKoL0"
      },
      "source": [
        "We demonstrated a similarity search approach for performing extreme classification of texts. We took a simple approach representing labels as the average of their corresponding texts' vector embeddings. In classification time, we match between a new article embedding and its nearest label embeddings. Our result examples indicate the usefulness of this approach. \n",
        "\n",
        "You can take this forward by exploring advanced ideas. For example, you can utilize the hierarchical relationship between labels or improve the label representations.  Just have fun, and feel free to [share](https://www.pinecone.io/contact/) your thoughts. "
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "Mbt5Ec7tKoL0"
      },
      "source": [
        "# Delete the index"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "9y6iIor1KoL0"
      },
      "source": [
        "Delete the index once you do not want to use it anymore. Once the index is deleted, you cannot use it again."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vAMnQjvsxSw0"
      },
      "outputs": [],
      "source": [
        "pc.delete_index(index_name)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "---"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "provenance": []
    },
    "environment": {
      "name": "common-cpu.m65",
      "type": "gcloud",
      "uri": "gcr.io/deeplearning-platform-release/base-cpu:m65"
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.7 (main, Sep 14 2022, 22:38:23) [Clang 14.0.0 (clang-1400.0.29.102)]"
    },
    "vscode": {
      "interpreter": {
        "hash": "b0fa6594d8f4cbf19f97940f81e996739fb7646882a419484c72d19e05852a7e"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 1
}